Table of Contents

When your Python project relies on external packages, you need to make sure you’re using the right version of each package. After an update, a package might not work as it did before the update. A dependency manager like Python Poetry helps you specify, install, and resolve external packages in your projects. This way, you can be sure that you always work with the right dependency version on every machine.

In this tutorial, you’ll learn how to:

Using Poetry will help you start new projects, maintain existing ones, and master dependency management. You’ll be prepared to work with pyproject.toml files, which will be the standard for defining build requirements in Python projects.

To complete this tutorial and get the most out of it, you should have a basic understanding of virtual environments, modules and packages, and pip.

While this tutorial focuses on dependency management, Poetry can also help you with building and packaging projects. If you want to share your work, then you can even publish your Poetry project to the Python Packaging Index (PyPI).

Free Bonus: Click here to get access to a free 5-day class that shows you how to avoid common dependency management issues with tools like Pip, PyPI, Virtualenv, and requirements files.

Before diving into the nitty-gritty of Python Poetry, you’ll take care of some prerequisites. First, you’ll read a short overview of terminology you’ll encounter in this tutorial. Next, you’ll install Poetry itself.

If you’ve ever used an import statement in one of your Python scripts, then you’ve worked with modules. Some of these modules might have been Python files you wrote on your own. Others could have been built-in modules, like datetime. However, sometimes what Python provides isn’t enough. That’s when you might turn to external, packaged modules. When your Python code relies on external modules, you can say that these packages are dependencies of your project.

You can find packages that aren’t part of the Python standard library in PyPI. Before seeing how this works, you need to install Poetry on your system.

To use Poetry in your command line, you should install it system-wide. If you just want to try it out, then you can install it into a virtual environment using pip. But you should try this method with caution because Poetry will install its own dependencies, which can conflict with other packages you’re using in your project.

The recommended way to install Poetry is by using the official install-poetry script. You can either download and run this Python file manually or select your operating system below to use the appropriate command:

If you’re on Windows, then you can use the Invoke-Webrequest cmdlet with the -UseBasicParsing option to download the content of a requested URL to the standard output stream (stdout). With the pipe character (|), you’re handing over the output to the standard input stream (stdin) of python. In this case, you’re piping the content of install-poetry.py to your Python interpreter.

Note: Some users report errors on Windows 10 when they use the PowerShell command.

With curl, you’re outputting the content of a requested URL to the standard output stream (stdout). By using a Unix pipeline with the pipe character (|), you’re handing over the output to the standard input stream (stdin) of python3. In this case, you’re piping the content of install-poetry.py to your Python interpreter.

Note: If you’re on macOS, then you may get an ssl.SSLCertVerificationError. This error can occur if you don’t have the default root certificates for the SSL module installed. You can install them by running a command script in your Python folder:

Depending on your installed Python version, the specific path to the Python interpeter might be different. In that case, you need to adjust the path in the above command accordingly.

After you run the command, the curl command above should work without any errors.

In the output, you should see a message that the installation is complete. You can run poetry --version in your terminal to see if poetry works. This command will display your current Poetry version. If you want to update Poetry, then you can run poetry self update.

With Poetry installed, it’s time to see how Poetry works. In this section, you’ll learn how to start a fresh Poetry project and how to add Poetry to an existing project. You’ll also see the project structure and inspect the pyproject.toml file.

You can create a new Poetry project by using the new command and a project name as an argument. In this tutorial, the project is called rp-poetry. Create the project, and then move into the newly created directory:

By running poetry new rp-poetry, you create a new folder named rp-poetry/. When you look inside the folder, you’ll see a structure:

Poetry automatically normalizes package names for you. It transformed the dash (-) in your project name into an underscore (_) in the folder name of rp_poetry/. Otherwise, the name wouldn’t be allowed in Python, so you couldn’t import it as a module. To have more control over creating the package name, you can use the --name option to name it differently than the project folder:

If you prefer to store your source code in an additional src/ parent folder, then Poetry lets you stick to that convention by using the --src flag:

By adding the --src flag, you’ve created a folder named src/, which contains your rp_poetry/ directory:

When creating a new Poetry project, you’ll receive a basic folder structure right away.

The rp_poetry/ subfolder itself isn’t very spectacular yet. Inside this directory, you’ll find an __init__.py file with your package’s version:

When you hop over to the tests/ folder and open test_rp_poetry.py, you’ll notice that rp_poetry is already importable:

Poetry has also added a first test to the project. The test_version() function checks whether the __version__ variable of rp_poetry/__init__.py contains the expected version. However, the __init__.py file isn’t the only place where you define the version of your package. The other location is the pyproject.toml file.

One of the most important files for working with Poetry is the pyproject.toml file. This file isn’t an invention of Poetry. It’s a configuration file standard that was defined in PEP 518:

This PEP specifies how Python software packages should specify what build dependencies they have in order to execute their chosen build system. As part of this specification, a new configuration file is introduced for software packages to use to specify their build dependencies (with the expectation that the same configuration file will be used for future configuration details). (Source)

The authors considered a few file formats for the “new configuration file” mentioned in the quote above. In the end, they decided on the TOML format, which stands for Tom’s Obvious Minimal Language. In their opinion, TOML is flexible enough, with better readability and less complexity than the other options, which are YAML, JSON, CFG, or INI. To see how TOML looks, open the pyproject.toml file:

You can see four sections in the pyproject.toml file. These sections are called tables. They contain instructions that tools like Poetry recognize and use for dependency management or build routines.

If a table name is tool-specific, it must be prefixed with tool. By using such a subtable, you can add instructions for different tools in your project. In this case, there is only tool.poetry. But you might see examples like [tool.pytest.ini_options] for pytest in other projects.

In the [tool.poetry] subtable on line 3 above, you can store general information about your Poetry project. Your available keys are defined by Poetry. While some keys are optional, there are four that you must specify:

The subtables [tool.poetry.dependencies] on line 9 and [tool.poetry.dev-dependencies] on line 12 are essential for your dependency management. You’ll learn more about these subtables in the next section when you add dependencies to your Poetry project. For now, the important thing is to recognize that there is differentiation between package dependencies and development dependencies.

The last table of the pyproject.toml file is [build-system] on line 15. This table defines data that Poetry and other build tools can work with, but as it’s not tool-specific, it doesn’t have a prefix. Poetry created the pyproject.toml file with two keys in place:

If you want to learn a bit more about this section of the pyproject.toml file, then you can find out more by reading about source trees in PEP 517.

When you start a new project with Poetry, this is the pyproject.toml file you start with. Over time, you’ll add configuration details about your package and the tools you’re using. As your Python project grows, your pyproject.toml file will grow with it. This is particularly true for the subtables [tool.poetry.dependencies] and [tool.poetry.dev-dependencies]. In the next section, you’ll find out how to expand these subtables.

Once you’ve set up a Poetry project, the real work can begin. You can start coding once Poetry is in place. Along the way, you’ll find out how Poetry provides you with a virtual environment and takes care of your dependencies.

When you start a new Python project, it’s good practice to create a virtual environment. Otherwise, you may confuse different dependencies from different projects. Working with virtual environments is one of Poetry’s core features, and it’ll never interfere with your global Python installation.

However, Poetry doesn’t create a virtual environment right away when you start a project. You can confirm that Poetry hasn’t created a virtual environment by having Poetry list all virtual environments connected to the current project. If you haven’t already, cd into rp-poetry/ and then run a command:

For now, there shouldn’t be any output.

Poetry will create a virtual environment along the way when you run certain commands. If you want to have better control over the creation of a virtual environment, then you might decide to tell Poetry explicitly which Python version you want to use for it and go from there:

With this command, you’re using the same Python version that you used to install Poetry. Using python3 works when you have the Python executable in your PATH.

Note: Alternatively, you can pass an absolute path to a Python executable. It should match the Python version constraint you can find in your pyproject.toml file. If it doesn’t, then you may run into trouble because you’re working with a different Python version than the version your project requires. Code that works in your environment could be buggy on another machine.

Even worse, external packages often rely on specific Python versions. Consequently, a user installing your package might get an error because your dependency versions are incompatible with their Python version.

When you run env use, you’ll see a message:

As you can see, Poetry constructed a unique name for your project’s environment. The name contains the project name and the Python version. The seemingly random string in the middle is a hash of your parent directory. With this unique string in the middle, Poetry can handle multiple projects with the same name and the same Python version on your system. That’s important because, by default, Poetry creates all your virtual environments in the same folder.

Without any other configuration, Poetry creates the virtual environments in the virtualenvs/ folder of Poetry’s cache directory:

If you want to change the default cache directory, then you can edit Poetry’s configuration. This can be useful when you’re already using virtualenvwrapper or another third-party tool for managing your virtual environments. To see the current configuration, including the configured cache-dir, you can run a command:

Usually, you don’t have to change this path. If you want to learn more about interacting with Poetry’s virtual environments, then the Poetry documentation contains a chapter about managing environments.

As long as you’re inside your project folder, Poetry will use the virtual environment associated with it. If you’re ever in doubt, you can check whether the virtual environment is activated by running the env list command again:

This will display something like rp-poetry-AWdWY-py3.9 (Activated). With an activated virtual environment, you’re ready to start managing some dependencies and see Poetry shine.

A key element of Poetry is its handling of your dependencies. Before you get the ball rolling, take a look at the two dependency tables in the pyproject.toml file:

There are currently two dependencies declared for your project. One is Python itself. The other is pytest, a widely used testing framework. As you’ve seen before, your project contains a tests/ folder and a test_rp_poetry.py file. With pytest as a dependency, Poetry can run your tests immediately after installation.

Note: At the time of writing this tutorial, running pytest with Poetry using Python 3.10 doesn’t work. Poetry installs a pytest version that is incompatible with Python 3.10.

The Poetry developers are aware of this issue, and it will be fixed with the release of Poetry 1.2.

Make sure that you’re inside the rp-poetry/ project folder and run a command:

With the install command, Poetry checks your pyproject.toml file for dependencies then resolves and installs them. The resolving part is especially important when you have many dependencies that require different third-party packages with different versions of their own. Before installing any packages, Poetry figures out which version of a package fulfills the version constraints that other packages set as their requirements.

Besides pytest and its requirements, Poetry also installs the project itself. This way, you can import rp_poetry into your tests right away:

With your project’s package installed, you can import rp_poetry into your tests and check for the __version__ string. With pytest installed, you can use the poetry run command to execute the tests:

Your current test is running successfully, so you can confidently continue coding. However, if you look closely at line 3, something looks a bit odd. It says pytest-5.4.3, not 5.2 like stated in the pyproject.toml file. Good catch!

To recap, the pytest dependency in your pyproject.toml file looks like this:

The caret (^) in front of 5.2 has a specific meaning, and it’s one of the versioning constraints that Poetry provides. It means that Poetry can install any version that matches the leftmost non-zero digit of the version string. This means that using 5.4.3 is allowed. Version 6.0 wouldn’t be allowed.

A symbol like the caret will become important when Poetry tries to resolve the dependency versions. If there are only two requirements, this isn’t too hard. The more dependencies you declare, the more complicated it gets. Let’s see how Poetry handles this by installing new packages into your project.

You may have used pip before to install packages that aren’t part of the Python standard library. If you run pip install with the package name as an argument, pip looks for packages on the Python Package Index. You can use Poetry the same way.

If you want to add an external package like requests to your project, then you can run a command:

By running poetry add requests, you’re adding the latest version of the requests library to your project. You can use version constraints like requests>>>>> import requests >>> import bs4 Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'bs4'

As expected, you can import requests without any trouble, and the module bs4 can’t be found. Exit the interactive Python interpreter by typing exit() and hitting Enter.

After locking dependencies with the poetry lock command, you have to run the poetry install command so that you can actually use them in your project:

By running poetry install, Poetry reads the poetry.lock file and installs all dependencies that are declared in it. Now, bs4 is ready for you to use in your project. To test this, enter poetry run python3 and import bs4 into the Python interpreter:

Perfect! No errors this time, and you have the exact version you declared. This means that Beautiful Soup is pinned correctly in your poetry.lock file, is installed in your project, and is ready to use. To list available packages in your project and inspect their details, you can use the show command. When you run it with the --help flag, you’ll see how to use it:

To inspect a package, you can use show with the package name as an argument, or you can use the --tree option to list all dependencies as a tree. This will help you see the nested requirements of your project.

For updating your dependencies, Poetry provides different options depending on two scenarios:

You can find your version constraints in your pyproject.toml file. When a new version of a dependency still fulfills your version constraints, you can use the update command:

The update command will update all your packages and their dependencies within their version constraints. Afterward, Poetry will update your poetry.lock file.

If you want to update one or more specific packages, then you can list them as arguments:

With this command, Poetry will search for a new version of requests and a new version of beautifulsoup4 that fulfill the version constraints listed in your pyproject.toml file. Then it’ll resolve all dependencies of your project and pin the versions into your poetry.lock file. Your pyproject.toml file will stay the same because the listed constraints remain valid.

If you want to update a dependency with a version that’s higher than the defined version in the pyproject.toml file, you need to adjust the pyproject.toml file beforehand. Another option is to run the add command with a version constraint or the latest tag:

When you run the add command with the latest tag, it looks for the latest version of the package and updates your pyproject.toml file. Including the latest tag or a version constraint is critical in using the add command. Without it, you’d get a message that the package is already present in your project. Also, don’t forget to add the --dev flag for development dependencies. Otherwise, you’d add the package to your regular dependencies.

After adding a new version, you must run the install command you learned about in the section above. Only then are your updates locked into the poetry.lock file.

If you’re not sure which version-based changes an update would introduce to your dependencies, you can use the --dry-run flag. This flag works for both the update and the add commands. It displays the operations in your terminal without executing any of them. This way, you can spot version changes safely and decide which update scenario works best for you.

While the version requirement in the pyproject.toml file can be loose, Poetry locks the versions you’re actually using in the poetry.lock file. That’s why you should commit this file if you’re using Git. By providing a poetry.lock file in a Git repository, you ensure that all developers will use identical versions of required packages. When you come across a repository that contains a poetry.lock file, it’s a good idea to use Poetry for it.

With poetry.lock, you can make sure that you use exactly the versions the other developers are using. And if the other developers aren’t using Poetry, you can add it to an existing project that wasn’t set up with Poetry.

Chances are, you have projects that you didn’t start with the poetry new command. Or maybe you inherited a project that wasn’t created with Poetry, but now you want to use Poetry for your dependency management. In these types of situations, you can add Poetry to existing Python projects.

If your project only contains some Python files, then you can still add Poetry as a foundation for future builds. In this example, there’s only one file, hello.py:

The only thing this script does is to output the string "Hello World!". But maybe this is just the beginning of a grand project, so you decide to add Poetry to your project. Instead of using the poetry new command from before, you’ll use the poetry init command:

The poetry init command will start an interactive session to create a pyproject.toml file. Poetry gives you recommendations for most of the configurations you need to set up, and you can press Enter to use them. When you don’t declare any dependencies, the pyproject.toml file that Poetry creates looks something like this:

The content looks similar to the examples that you went through in the previous sections.

Now you can use all the commands that a Poetry project offers. With a pyproject.toml file present, you can now run scripts:

Because Poetry didn’t find any virtual environments to use, it created a new one before executing your script. After doing this, it displays your Hello World! message without any errors. That means you now have a working Poetry project.

Sometimes you have projects that already have a requirements.txt file. Take a look at the requirements.txt file of this Python web scraper:

With the cat utility, you can read a file and write the content to the standard output. In this case, it shows the dependencies of the web scraper project. Once you’ve created the Poetry project with poetry init, you can combine the cat utility with the poetry add command:

When a requirements file is straightforward like this, using poetry add and cat can save you some manual work.

Sometimes requirements.txt files are a bit more complicated, however. In those cases, you can either execute a test run and see how it turns out or add requirements by hand to the [tool.poetry.dependencies] table in the pyproject.toml file. To see if the structure of your pyproject.toml is valid, you can run poetry check afterward.

In some situations, you must have a requirements.txt file. For example, maybe you want to host your Django project on Heroku. For cases like this, Poetry provides the export command. If you have a Poetry project, you can create a requirements.txt file from your poetry.lock file:

Using the poetry export command in this way creates a requirements.txt file that includes hashes and environment markers. This means that you can be sure to work with very strict requirements that resemble the content of your poetry.lock file. If you also want to include your development dependencies, you can add --dev to the command. To see all available options, you can check poetry export --help.

This tutorial has introduced you to Poetry’s dependency management. Along the way, you’ve used some of Poetry’s command-line interface (CLI) commands:

You can check out the Poetry CLI documentation to learn more about the commands above and the other commands Poetry offers. You can also run poetry --help to see information right in your terminal!

In this tutorial, you explored how to create a new Python Poetry project and how to add Poetry to an existing one. A key part of Poetry is the pyproject.toml file. In combination with poetry.lock, you can ensure that you install the exact version of each package that your project requires. When you track the poetry.lock file in your Git repository, you also make sure that all other developers in the project install the same dependency versions on their machines.

In this tutorial, you learned how to:

This tutorial focused on the basics of Poetry’s dependency management, but Poetry can also help you build and upload your package. If you want to get a taste of this capability, then you can read about how to use Poetry when publishing an open source Python package to PyPI).

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

About Philipp Acsany

Philipp is a Berlin-based software engineer with a graphic design background and a passion for full-stack web development.

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Aldren

Bartosz

Geir Arne

Kate

Martin

Sadie

Master Real-World Python Skills With Unlimited Access to Real Python

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python SkillsWith Unlimited Access to Real Python

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.

Keep Learning

Related Tutorial Categories: best-practices devops intermediate tools

Continue »

Already have an account? Sign-In